Modified Files:
src/render.cpp (for generating variance map>
src/main.cpp
scenes/denoising/python/moderate denoising (independent variance estimate) (python code, for denoising)
Validation Files:
image without noise: scenes/denoising/cbox/cbox_path_mats_16384.exr
image with noise: scenes/denoising/cbox/cbox_path_mats_512.exr
image variance: scenes/denoising/cbox/cbox_path_mats_512_var_var.exr
denoised image: scenes/denoising/python/denoised_xxx.exr
Implementation
Our implementation follows the patchwise implementation algorithm of Non-Local Means Denoising and the pesudo code introduced in the course slide.
Instead of generating one pixel's value completely then going to the other pixel, the fast implementation at each time only considers the contributions of patches pairs with same offsets. The first average convolution compute the difference map between two pacthes centered around the pixel. Since the contribution is patch-wise, a pixel could be contributed by multiple same-offset patch pairs as long as it's within the patch scope. This contribution is computed by the second average convolution on the difference map. The final value for each pixel is the weighted sum divided by weights.
The original paper uses the uniform variance for all pixels, we improve this by estimating pixel-wise variance during the rendering. The sample mean variance estimate algorithm is Welford's online algorithm, which is introduced in the course slides. To achieve this, auxillary data structures storing one sample for each image pixel, mean, mean2 are created.
Implementation Details:
1. denoising is performed on Python, image and image variance are stored as numpy array and convolution operator is from scipy.
2. for image boundary handling, I tried wrap(circular), mirror and nearest padding method. The results produced have negligible differences (only visible by tev comparing mode).
3. scipy.signal.convolve() and scipy.ndimage.convolve() both provides convolution operations. The difference is that the former has much higher efficiency but supports only 2D conv and limited padding, the latter supprots diverse padding and nD conv but the efficiency is low. I implemented both versions.
4. EXR file reading and writing requires opencv, OpenEXR, Imath packages.
5. The render doesn't export mean variance image by default, to enable the function, ./nori xxx.xml require_varmap true
Validation
Noisy Image and Variance
The noisy image is generated by 512 sampling with MATS integrator. To verify the correctness of the rendered image and its corresponding pixel mean variance image, I visualize them on the jupyter notebook. The original image is transformed by gamma function for visualization, the mean variance image is x512 for visualization.
ingredients for denoising
Comparison: noisy, denoised, noise-free
The noise-free image is generated by 16384 samples with MATS integrator. The padding mode for the selected denoised image is 'wrap'(a.k.a circular) because it have the best efficiency. The parameters r, f, k for denoising are 10, 3, 0.45 respectively.
Comparison Between Denoised and Noise-Free Images
Comparison: different boundary padding modes
By tev pixel-wise comparsion, the denoised images are almost same under different padding modes and have no visual differences.
Comparison on different padding modes